A Neuro-Symbolic ASP Pipeline for Visual Question Answering

Abstract We present a neuro-symbolic visual question answering (VQA) pipeline for CLEVR, which is well-known dataset that consists of pictures showing scenes with objects and questions related to them. Our covers (i) training neural networks object classification bounding-box prediction the CLEVR scenes, (ii) statistical analysis on distribution values determine threshold high-confidence predictions, (iii) translation network predictions pass confidence thresholds into logic programmes so we can compute answers using an answer-set programming solver. By exploiting choice rules, consider deterministic non-deterministic scene encodings. experiments show encoding achieves good results even if are trained rather poorly in comparison approach. This important building robust VQA systems less-than perfect. Furthermore, restricting non-determinism reasonable choices allows more efficient implementations approaches without losing much accuracy.